Dead & Company Tour Analysis

Overview

Born from the ashes of their iconic predecessors, Dead & Company channels the heart and soul of the Grateful Dead, captivating legions of devoted Deadheads worldwide. Their awe-inspiring performances have woven an eternal bond with their ardent fans, who have traversed time and space to follow their melodic odyssey.Drawing upon the boundless potential of R’s data manipulation prowess, we unfurl the tour setlists spanning seven captivating summers. Our quest begins with dynamic leaflet maps that whisk you away on a virtual journey, guiding you through the transcendent venues and harmonious landscapes where Dead & Company’s symphonic magic danced under the stars. Each concert is a celestial point, forming an enchanting constellation of memories etched into the fabric of American soil. Venturing deeper, we illuminate the band’s timeless anthems through insightful bar graphs that paint a vivid portrait of the most frequented melodies. Witness the sonic phenomena that have etched themselves into the hearts of Deadheads, resonating with an ineffable power that defies the hands of time. And then, we unveil a majestic choropleth map, a kaleidoscopic tapestry of colors dancing across the map of the United States. Each hue represents the frequency of concerts in every state, a testament to the band’s unwavering connection with fans spanning the nation.The journey reaches its poetic crescendo with an enchanting poster that summarizes our journey using maps, word clouds, bar graphs, and the uncovering of the secret encores and grand openings that crowned each summer’s cosmic expedition. Marvel at the song selection sorcery that left fans spellbound, elevating the euphoria of each concert into a breathtaking crescendo. each a lyrical constellation, adorning the cosmos with the most-played songs from every celestial tour. Behold the poetic brilliance of lyrics woven into captivating visuals, where the essence of Dead & Company’s music shines with unparalleled brilliance.

Prepare to embark on a thrilling adventure as we delve into the extraordinary world of Dead & Company’s enchanting summer tours! With boundless enthusiasm, we unlock the magic of their musical journey through a mesmerizing tapestry of interactive maps, mesmerizing bar graphs, ethereal word clouds, and captivating choropleth maps. Powered by the magic of R and an arsenal of data visualization wizardry, we embark on an awe-inspiring exploration of the band’s illustrious touring history from the awe-inspiring year of 2016 to the spellbinding realm of 2023.

If you would like to play around with things we cover or try something new all the data used in this project is available to download HERE

Part I. Data Collection and Preprocessing

The foundation of our project rests on meticulous data collection, where we obtained tour setlists for each summer from 2016 to 2023. These datasets were processed using R’s data manipulation capabilities, specifically the tidyverse package, allowing us to clean, filter, and transform the data into a format suitable for visualization.

In this section, we unveil the cornerstone of our project—meticulous data collection and pre-processing. Our journey commences with the acquisition of tour setlists for each summer spanning from 2016 to 2023. Armed with the might of R’s data manipulation prowess, we harness the power of the tidyverse package to skillfully clean, filter, and transform the datasets, molding them into a format primed for captivating visualizations.

Because I was using an already established geo-database it did not contain certain cities that we need so I had to create new shapefiles of missing points, give them unique ID codes, and filter out repeating cities. It got quite messy so I will not include that here as it would make this document much longer but everything I gloss over here is available in the projects zip file.

Part II. Interactive Maps with Leaflet

Now, with our data primed and ready, we dive deep into the enchanting world of Dead & Company’s captivating summer tour of 2016. Leverage the power of interactive mapping through the Leaflet package, we traverse the musical tapestry woven across the nation’s landscapes. Marvel at the radiant constellation of concerts—each a celestial point, emblematic of a moment where magic unfolded beneath the stars.

This is only the beginning of our spellbinding expedition, as we venture onward to explore the mystique of subsequent summers, basking in the euphoric glory that continues to bind Dead & Company with their ever-devoted Deadheads. Let us embrace the essence of their legacy, a fusion of data and artistry that reverberates with the timeless magic of music.

For the sake of keeping this brief I will share how I made the leaflet 2016 tour map. Every other map is pretty rinse and repeat so it shouldn’t be too difficult for yourself to make a leaflet map of other tour years. Or if you don’t want to make your own you can find leaflet maps inside of the maps folder inside of the project zip file.

tour16 <- read.csv("summer_tours/summer2016.csv")

summer16_cities <- cities %>%
  filter(FID_1 == 2264|FID_1 == 3511|FID_1 == 2382|FID_1 == 301|
           FID_1 == 3405|FID_1 == 2215|FID_1 == 9998|FID_1 == 2174|
           FID_1 == 726|FID_1 == 661|FID_1 == 9996|FID_1 == 9994|
           FID_1 == 9997|FID_1 == 1437|FID_1 == 2913|FID_1 == 9999|
           FID_1 == 1165|FID_1 == 1052|FID_1 == 9989|FID_1 == 452)
summer16_cities$FID_1[15:19] <- 0

summer16_cities <- summer16_cities %>%
  filter(!FID_1 == 0)

tour16$Show_num <- as.factor(tour16$Show_num)
levels(tour16$Show_num)

##  [1] "1"  "2"  "3"  "4"  "5"  "6"  "7"  "8"  "9"  "10" "11" "12" "13" "14" "15"
## [16] "16" "17" "18" "19" "20" "21" "22" "23" "24"

setlists16 <- tour16 %>%
  group_by(Show_num) %>%
  summarize(setlist = paste(Song, collapse = "<br>"))

tour16_with_geometry <- left_join(tour16, summer16_cities[, c("FID_1", "geometry")], by = "FID_1")

# Extract latitude and longitude from geometry column
tour16_with_geometry$latitude <- st_coordinates(tour16_with_geometry$geometry)[, 2]
tour16_with_geometry$longitude <- st_coordinates(tour16_with_geometry$geometry)[, 1]

tour16_with_geometry <- tour16_with_geometry[complete.cases(tour16_with_geometry$latitude, tour16_with_geometry$longitude), ]

# Convert tour18_with_geometry to sf object
tour16_with_geometry_sf <- st_as_sf(tour16_with_geometry, coords = c("longitude", "latitude"), crs = 4326)

# Transform the sf object to WGS84
tour16_with_geometry_sf <- st_transform(tour16_with_geometry_sf, crs = 4326)

# Group the songs by FID_1 and concatenate them into a single string
setlist16 <- tour16_with_geometry_sf %>%
  group_by(FID_1) %>%
  summarize(setlist = paste(Song, collapse = "<br>"))

# Merge the setlist data with the tour23_with_geometry_sf
tour16_with_setlist <- st_join(tour16_with_geometry_sf, setlist16)

This is a critical step as for whatever reason while adding and dropping geometry some cities and their setlists are lost so its key that we get those back. This happened for Boston and Chicago almost every time.

# Missing cities; Boston, Portland, George, Wheatland
# Boston
missing_bos16 <- setlist16 %>%
  filter(FID_1 == 1437)
missing_bos16_df <- as.data.frame(missing_bos16)
tour16_with_setlist <- tour16_with_setlist %>%
  dplyr::left_join(missing_bos16_df[, c("FID_1", "setlist")], by = c("FID_1.x" = "FID_1")) %>%
  dplyr::mutate(setlist = ifelse(!is.na(setlist.y), setlist.y, setlist.x)) %>%
  dplyr::select(-setlist.x, -setlist.y)
# Portland
missing_por16 <- setlist16 %>%
  filter(FID_1 == 2913)
missing_por16_df <- as.data.frame(missing_por16)
tour16_with_setlist <- tour16_with_setlist %>%
  dplyr::left_join(missing_por16_df[, c("FID_1", "setlist")], by = c("FID_1.x" = "FID_1")) %>%
  dplyr::mutate(setlist = ifelse(!is.na(setlist.y), setlist.y, setlist.x)) %>%
  dplyr::select(-setlist.x, -setlist.y)
# George
missing_geo16 <- setlist16 %>%
  filter(FID_1 == 9999)
missing_geo16_df <- as.data.frame(missing_geo16)
tour16_with_setlist <- tour16_with_setlist %>%
  dplyr::left_join(missing_geo16_df[, c("FID_1", "setlist")], by = c("FID_1.x" = "FID_1")) %>%
  dplyr::mutate(setlist = ifelse(!is.na(setlist.y), setlist.y, setlist.x)) %>%
  dplyr::select(-setlist.x, -setlist.y)
# Wheatland
missing_whe16 <- setlist16 %>%
  filter(FID_1 == 9989)
missing_whe16_df <- as.data.frame(missing_whe16)
tour16_with_setlist <- tour16_with_setlist %>%
  dplyr::left_join(missing_whe16_df[, c("FID_1", "setlist")], by = c("FID_1.x" = "FID_1")) %>%
  dplyr::mutate(setlist = ifelse(!is.na(setlist.y), setlist.y, setlist.x)) %>%
  dplyr::select(-setlist.x, -setlist.y)


tour16_with_setlist$latitude <- st_coordinates(tour16_with_setlist$geometry)[, 2]
tour16_with_setlist$longitude <- st_coordinates(tour16_with_setlist$geometry)[, 1]

map16 <- leaflet() %>%
  addTiles(group = "OSM") %>%
  addProviderTiles("Esri.WorldGrayCanvas", group = "ESRI") %>%
  addProviderTiles("CartoDB.DarkMatter", group = "CartoDB") %>%
  addPolygons(data = states, color = "grey", fill = FALSE, weight = 2,
              highlight = highlightOptions(weight = 3, color = "white", bringToFront = FALSE), popup = FALSE) %>%
  addCircleMarkers(data = tour16_with_setlist, radius = 8, stroke = FALSE, fillOpacity = 1,
                   lat = ~latitude, lng = ~longitude, popup = ~setlist, group = "Tour16",
                   color = color_palette[as.numeric(factor(tour16_with_setlist$Location))]) %>%
  setView(lat = 39, lng = -100, zoom = 4) %>%
  addLayersControl(
    baseGroups = c("OSM", "ESRI", "CartoDB"),
    options = layersControlOptions(collapsed = FALSE))

encore_songs16 <- tour16_with_setlist[tour16_with_setlist$Set.Type == "Encore", ]
encore_songs16 <- sf::st_drop_geometry(encore_songs16)

encore_agg16 <- encore_songs16 %>%
  group_by(Location) %>%
  summarize(encore_setlist = paste(Song, collapse = "<br>"))

# Merge aggregated encore songs with tour16_with_setlist
tour16_with_setlist <- left_join(tour16_with_setlist, encore_agg16, by = "Location")

map16 <- map16 %>%
  addCircleMarkers(data = tour16_with_setlist, radius = 8, stroke = FALSE, fillOpacity = 1,
                   lat = ~latitude, lng = ~longitude, popup = ~encore_setlist, group = "Encore Songs",
                   color = encore_palette)

# Add layers control
map16 <- map16 %>%
  addLayersControl(
    overlayGroups = c("Tour16", "Encore Songs"),
    options = layersControlOptions(collapsed = FALSE),
    baseGroups = c("OSM", "ESRI", "CartoDB"),
    position = "topright"
  )

opening_songs16 <- tour16_with_setlist[tour16_with_setlist$Song.Order == 1, ]
opening_songs16 <- sf::st_drop_geometry(opening_songs16)

opening_agg16 <- opening_songs16 %>%
  group_by(Location) %>%
  summarize(opening_setlist = paste(Song, collapse = "<br>"))

tour16_with_setlist <- left_join(tour16_with_setlist, opening_agg16, by = "Location")

map16 <- map16 %>%
  addCircleMarkers(data = tour16_with_setlist, radius = 8, stroke = FALSE, fillOpacity = 1,
                   lat = ~latitude, lng = ~longitude, popup = ~opening_setlist, group = "Opening Songs",
                   color = opening_palette)
map16 <- map16 %>%
  addLayersControl(
    overlayGroups = c("Tour16", "Encore Songs", "Opening Songs"),
    options = layersControlOptions(collapsed = FALSE),
    baseGroups = c("OSM", "ESRI", "CartoDB"),
    position = "topright"
  )

And there you have it! An interactive map that contains all shows from the Summer 2016 tour and Layer controls to filter though the entire setlist! There are also layer controls for the base-map as well so you can get it to suit your visual preferences.

map16

Part III. Vector Based Analysis Using tmap

Once again we have some annoying data cleaning to do, as it is quite intensive I have decided to spare you from scrolling through it.

Now that our data is cleaned up we can start to manipulate some things and get all of our required data in the right place.

Using tmap and our aggregated data we can finally use tmap to visualize all the places Dead & Company have visited over the past 7 summers!

dead_tmap <- tm_shape(merged_states) +
  tm_polygons(col = "num_shows", palette = "Reds", title = "", legend.show = FALSE) +
  tm_shape(merged_states) +
  tm_fill(col = "num_shows", palette = "Reds", 
  title = "# of concerts", legend.is.portrait = FALSE) +
  tm_shape(states) +
  tm_borders(col = "#000000", lwd = 1) +
  tm_shape(unique_cities_counts) +
  tm_bubbles(size = 0.25, col = "#8A2BE2") +
  tm_layout(legend.outside.position = "bottom",legend.outside.size = 0.35,legend.outside = TRUE, 
            main.title = "Seven Summers of Dead & Company",
            main.title.position = "center", 
            main.title.fontface = "bold",
            legend.text.size = 0.8)

dead_tmap

Part IV. Wordcloud

Time to set up a vibrant color palette cloud_palette for word clouds, which will be utilized to visually represent song frequencies from different Dead & Company summer tours.

This code chunk filters out Drums & Space. The reason being that it is performed at every show and would clearly result in being the largest word in our cloud so we’ll just take those out (sorry Mickey).

Next I calculate song frequency for each tour. Once again this is very rinse and repeat so it will not be included here but you can get an idea of what I did for each tour in the next code chunk which is more condense than this.

The code combines song frequency data from multiple years into a single data frame (all_song_freq). It then aggregates the total frequency for each song and sorts the data frame in descending order based on the total frequency.

#Combine song frequency data from multiple years
all_song_freq <- rbind(song_freq16, song_freq17, song_freq19,song_freq21, song_freq22, song_freq23)
# Aggregate total frequency for each song
total_freq <- aggregate(n ~ Song, data = all_song_freq, sum)
# Sort the data frame by total frequency in descending order
total_freq <- arrange(total_freq, desc(n))

Finally, this segment generates an impressive word cloud (total_cloud) representing the combined song frequencies from all the summer tours. The word cloud visually highlights the most frequently performed songs throughout the years. The color palette color_palette2 adds visual appeal to the word cloud, capturing the essence of Dead & Company’s diverse musical repertoire.

color_palette2 <- c("#0000FF", "#E7298A", "#E63946", "#00BFFF", "#1D3557", "#32CD32",
                    "#F3722C", "#FF6F91", "#9A031E", "#03A696", "#06D6A0", "#FF5733",
                    "#C70039", "#900C3F", "#8A2BE2", "#1287A5", "#F09727")

max_words2 <- 156

total_cloud <- wordcloud(words = total_freq$Song[1:max_words2], freq = total_freq$n[1:max_words2],
                         scale = c(1.5, 0.3), colors = color_palette2, bg = "white", rot.per = 0.2,
                         random.order = FALSE, min.freq = 1, max.words = max_words2,
                         random.color = TRUE, rotate.min = -45, rotate.max = 45,
                         ordered.colors = FALSE, use.r.layout = FALSE, asp = 0.75,
                         family = "plain", par.settings = list(cex = 0.8, col = "darkblue"))

After doing a little data exploration of my own I decided to at the very least share what I’ve found: All songs contain links to Youtube videos of their respective performances

Top 10 Most Frequent Songs

“China Cat Sunflower” - 38 times

“Deal” - 38 times

“I Know You Rider” - 38 times

“Bertha” - 37 times

“Brown Eyed Women” - 37 times

“Fire on the Mountain” - 37 times

“Franklin’s Tower” - 37 times

“Scarlet Begonias” - 37 times

“Playin in the Band” - 36 times

“Althea” - 35 times - (Johnny Kills It On This One)

Rare Songs:

“Maggie’s Farm” - 1 time

“My Favorite Things” - 1 time

“In The Midnight Hour” - 2 times

“Deep Elem Blues” - 2 times

“Death Dont Have No Mercy” - 3 times

“Promised Land” - 3 times

“Candyman” - 4 times

“Little Red Rooster” - 4 times

“Dont Ease Me In” - 5 times

“Man Smart, Women Smarter” - 5 times

Part V. Barchart

# Create the bar plot with custom fill colors
dead_plot <- ggplot(top_n_songs, aes(x = reorder(Song, n), y = n, fill = factor(n))) +
  geom_bar(stat = "identity") +
  scale_fill_manual(values = occurrences_colors) +  # Apply custom fill colors
  labs(x = "Song", y = "", title = "Top 50 Songs",
       subtitle = "Across All Seven Tours") +
  theme_minimal() +
  theme(axis.text.x = element_text(angle = 45, hjust = 1, vjust = 1),
        plot.title = element_text(size = 18, face = "bold"),
        plot.subtitle = element_text(size = 14, face = "italic"))

dead_plot

We have explored the frequency of encore performances and opening songs in Dead & Company’s summer tours. We extracted data on songs played during encores and those chosen to kick start the musical journey during each concert. These insights provide a comprehensive view of the band’s song selection and their emphasis on creating memorable experiences for their audiences. The data presented here unveils the diversity and magic that is woven into each performance.

## 'data.frame':    190 obs. of  2 variables:
##  $ Song    : chr  "Touch of Grey" "US Blues" "Black Muddy River" "Ripple" ...
##  $ Set.Type: chr  "Encore" "Encore" "Encore" "Encore" ...

## `summarise()` has grouped output by 'Song'. You can override using the
## `.groups` argument.

head(all_encore)

## # A tibble: 6 × 3
## # Groups:   Song [6]
##   Song                 Set.Type times_played
##   <chr>                <chr>           <int>
## 1 Althea               Encore              1
## 2 Black Muddy River    Encore             20
## 3 Brokedown Palace     Encore             25
## 4 Casey Jones          Encore              3
## 5 Deal                 Encore              1
## 6 Fire on the Mountain Encore              1

## 'data.frame':    165 obs. of  2 variables:
##  $ Song      : chr  "Music Never Stopped" "Truckin" "Hell in a Bucket" "Minglewood Blues" ...
##  $ Song.Order: int  1 1 1 1 1 1 1 1 1 1 ...

## `summarise()` has grouped output by 'Song'. You can override using the
## `.groups` argument.

head(all_openers)

## # A tibble: 6 × 3
## # Groups:   Song [6]
##   Song                Song.Order times_played
##   <chr>                    <int>        <int>
## 1 A Love Supreme               1            1
## 2 Althea                       1            1
## 3 Bertha                       1           12
## 4 Cassidy                      1            3
## 5 China Cat Sunflower          1            2
## 6 Cold Rain and Snow           1            8

As we traversed the musical tapestry woven across the nation’s landscapes, we witnessed the enduring power and brilliance of Dead & Company. Their captivating performances, born from the heart and soul of the Grateful Dead, have forged an eternal bond with devoted Deadheads worldwide. The band’s ability to create enchanting musical experiences, each concert a celestial point in the constellation of memories, speaks to their exceptional talent and enduring legacy.

Our love letter to Dead & Company’s legacy would not have been possible without the ardent fans and music enthusiasts who joined us on this thrilling adventure. To those who took the time to explore with us, we extend our deepest gratitude. Your passion for music and dedication to the magic of Dead & Company inspired us throughout this journey. As we bid farewell to this captivating exploration, we carry with us the joy of music, the allure of data, and the harmony that transcends time and space.

Thank you for joining us on this exhilarating escapade! Your presence has made this journey all the more enchanting, and we hope the magic of Dead & Company’s music continues to resonate with you for years to come. As we celebrate their legacy, may the melodies and spirit of Dead & Company ignite a timeless flame in the hearts of music lovers everywhere.

Farewell, the magic of music awaits, and we eagerly anticipate the harmonious adventures that lie ahead. Let us continue to cherish the melodies that have touched our souls and celebrate the enduring legacy of Dead & Company with boundless enthusiasm.

With deepest gratitude,

DC

Part VI. Aknowledgements, Data Sources, & Useful Links

Download this project HERE

Data for tours was sourced from the Dead & Company Database

Spatial data and skills were learned from West Virginia View

Another great resource for learning R in a different context can be found HERE

One of my favorite sites that contains (maybe?) the largest access to Grateful Dead shows can be found HERE

Connect with me on Twitter, LinkedIn, and Github